Wanted: live leak detection for FastMM
One of the awesome things about FastMM is FullDebugMode. Its features pretty much make memory leaks trivial to track down… usually. But if you’ve got the classic problem of code creating objects and placing them in a container somewhere and then abandoning them, you can have a huge memory leak in your program without it ever being detected, just as long as the container gets properly cleaned up before the program shuts down.
Seems to me that it should be possible to detect this with a bit of instrumenting, though. Here’s the basic idea.
When FastMM starts, it gets a brand new heap with nothing allocated in it yet, and it sets up its bookkeeping.
When FastMM shuts down, it checks to see if anything is still allocated in that heap. If so, it reports it.
If you want to detect “live leaks,” just apply that principle recursively: Run some routine called “LiveLeakCheckBegin” which allocates a new heap and some new bookkeeping for it. From this point on, all new allocations should go into this new heap.
At some later point, run “LiveLeakCheckEnd”. When this is run, FastMM would go through and generate a leak report based on everything still allocated *in the new heap*.
Obviously, this would only be for debugging purposes, and should not be used in a release build. Maybe it could be added to the FullDebugMode DLL as a new feature?
Pierre, if you’re reading this, how difficult would it be to implement something like this in FastMM?
Anyone else, do you have any ideas on how to implement it without having to modify FastMM?
I think that this will work only for very simple cases. Memory in real application usually do not follow simple patterns. Think about reference counting, owners, and multi-threading. You probably should get a lot of false-positive leaks.
However, FastMM4 already have basic necessary tools to troubleshoot such issues. The idea is to run your application until it almost run out of memory (i.e. consumes a bit less than 2 Gb for typical 32-bit app), now call LogAllocatedBlocksToFile to dump allocated memory to file. Open this file and pick any block. In most cases the chances that you hit your hidden leak are above 90%. You may pick two or 3 additional reports just to be sure.
Hello… hope this isn’t considered spam 😉 My DDDebug Memory Profiler does support showing the allocated memory (objects, strings, records etc.)) life – in real time. It has snapshots, which might be very similar to what to asked for. You can download a trial version here: http://ddobjects.de/dddebug
Here is what I have done to fix a live leak in one of our services:
– in the latest FastMM4 version, you can call “LogMemoryManagerStateToFile”. This functions dump a summary of all classes. When you call this functions 2 times, you can compare the differences with for example winmerge (which class count has been increased?)
– in my case, not classes where causing the live leak, but I got increasing string counts (using the above diff)
So I had to find out which strings where added between the 2 moments. I used “LogAllocatedBlocksToFile” for this:
– first store current group number: “GetCurrentAllocationGroup”
– increase this number and call “PushAllocationGroup(new group)”
– all new allocs get a different group number
– the second time you can dump a detailed memory dump of only these new allocs using “LogAllocatedBlocksToFile(new group)”
Note: I changed “DebugReallocMem” a bit, so it did not reuse the old allocation number.
This way I found out I was appending a string (for logging) but it got never cleared…
Hope you can use this technique too?
Good thinking, that’s something that would be nice.
Funny timing 😉
[…] TURBU Tech » Blog Archive » Wanted: live leak detection for FastMM. […]
You might have a look at http://ww2.continuit.nl/temp/UnitMemoryTester.pas. In this unit I wrap the current memory manager and you can get snapshots of the current memroy usage. For each allocation + callstack + threadID you get the # of allocations and the # of bytes. On top of this unit I have a unit that queries this information once every minute and dumps the information (and delta) into text files. Note that the unit only replaces the memmanager if next to the .exe there is a .mem file with the same name (which can be empty). This way I was able to leave this code inside the release version and when there is an unexpected memory leak that you cannot find, you restart the application after creating the .mem file and you have all the information you can get from a testrun in production without releasing additional executables. The reason I created it was exactly the case you describe above: a resource leak that was not a classic memory leak but a data container being filled and filled and filled (and never emptied).
The unit also places guard bytes around your memory allocations, fills the content with garbage on free so you see the use after free more easily etc etc. I use it together with FastMM and AQTime.
@Ritsaert Hornstra
Would you mind providing bit more info about how you use this unit at
http://ww2.continuit.nl/temp/UnitMemoryTester.pas.
I understand other unit calls some methods from this one. When and which methods?
Would you mind providing a sample of other unit?
Much appreciated,
db